Image processing device, image processing method,program, and recording medium

ABSTRACT

An image receiving unit receives an input of an image set owned by a first user, an image analyzing unit analyzes each image included in the image set, a tag information setting unit sets tag information items to be assigned to each image based on an analyzing result of each image, and a tag information assigning unit assigns, as main tag information, the tag information, among the tag information items to be assigned to the image, for which a ratio of the number of times of appearances of the tag information to be assigned to the image to the total number of times of appearances of all the tag information items to be assigned to all the images included in the image set is equal to or greater than a first threshold value and is equal to or less than a second threshold value, to the image, for each image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. § 119 toJapanese Patent Application No. 2018-184349, filed on Sep. 28, 2018. Theabove application(s) is hereby expressly incorporated by reference, inits entirety, into the present application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an image processing device, an imageprocessing method, a non-transitory computer recording medium storing aprogram, and a recording medium that assign tag information to eachimage included in an image set.

2. Description of the Related Art

An image owned by a user is published to any user who uses a socialnetworking service (SNS) or is shared with only a sharing partner byusing the SNS. In this case, tag information may be assigned to eachimage. The tag information is used for searching for and displaying adesired image from an image set and is used as information indicatingcontents of the image in association with the image.

The tag information can be automatically assigned by a program such asthe SNS or can be manually assigned by each user according to a subjectappearing in each image. One or more tag information items can beassigned to each image.

For example, tag information items such as “vehicle”, “train”, and “4000series train” are assigned to an image in which a train appears. Forexample, in a case where a girl unexpectedly appears in the image inwhich the train appears, tag information items such as “person”,“child”, and “girl” are assigned. As stated above, various taginformation items may be assigned to each image according to the subjectappearing in the image regardless of an importance of the subject.

Here, the related arts related to the present invention includeJP2017-059101A, JP2017-068434A, and JP2010-250635A.

JP2017-059101A describes that an image is obtain and analyzed, taginformation is assigned to the image based on analyzing result, anassignment rate of common tag information in images owned by a pluralityof users is calculated as a first assignment rate, an assignment rate ofthe common tag information in the image owned by each user is calculatedas a second assignment rate, and ranking of the common tag informationassigned to the image owned by each user is determined based on adifference between the first assignment rate and the second assignmentrate of the common tag information.

JP2017-068434A describes that a master-slave relation value in a casewhere a master-slave relation between two subjects is determined and thesubjects have a relation in which a first subject is a slave and asecond subject is a master is calculated as a value larger than amaster-slave relation value in a case where the subjects have notrelation in which the first subject is the slave and the second subjectis the master, a smaller master-slave relation value is calculated asthe number of subjects, among the plurality of subjects, which use thefirst subject as the slave becomes larger, and a subject evaluationvalue is calculated for each subject based on the master-slave relationvalue.

JP2010-250635A describes that image data is obtained and is stored in amemory, a plurality of feature values is obtained from the image datastored in the memory, and an importance in each feature value isdetermined. JP2010-250635A describes that a histogram is created, afrequency of a value is obtained, and the value having a high frequencyis determined as a value having high importance in a case where afeature value is a feature value capable of being obtained as anumerical value, as a determining procedure of the importance.

SUMMARY OF THE INVENTION

In recent years, smartphones have come into wide use. The number ofusers who display browse a list of image sets published or shared in theSNS on a display of the smartphone has increased. However, a displayregion of the display of the smartphone is relatively narrow, and thus,it may be difficult to display all tag information items assigned toeach image included in the list of images on the display in associationwith the image.

In a case where the image set is displayed, it is necessary to displaytag information appropriate to each image according to contents of theimage set. For example, in a case where the user narrows down images inwhich trains appear among the image set and displays the images, theuser knows that the trains appear in all the images. In this case, it isdesirable that tag information of “4000 series train” is displayedinstead of displaying tag information of “train”.

As stated above, in the related art, various tag information items canbe assigned to the image according to the subject appearing in theimage. However, in a case where the image included in the image set isdisplayed, the user cannot display the appropriate tag information inassociation with each image.

Accordingly, an object of the present invention is to provide an imageprocessing device, an image processing method, a non-transitory computerrecording medium storing a program, and a recording medium which arecapable of displaying tag information appropriate to each image inassociation with each image in a case where a list of images included inan image set is displayed.

In order to achieve the object, the present invention provides an imageprocessing device comprising an image receiving unit that receives aninput of an image set owned by a first user, an image analyzing unitthat analyzes each image included in the image set, a tag informationsetting unit that sets tag information items to be assigned to eachimage based on analyzing result of each image, and a tag informationassigning unit that assigns, as main tag information, the taginformation, among the tag information items to be assigned to theimage, for which a ratio of the number of times of appearances of thetag information to be assigned to the image to the total number of timesof appearances of all the tag information items to be assigned to allthe images included in the image set is equal to or greater than a firstthreshold value and is equal to or less than a second threshold value,to the image, for each image.

Here, it is preferable that the tag information setting unit sets, asthe tag information items to be assigned to the image, two or more taginformation items which have different abstraction degrees, among aplurality of tag information items which is included in the samecategory and has different abstraction degrees, for each image, and thetag information assigning unit assigns, as the main tag information, taginformation, among the two or more tag information items which are to beassigned to the image and have different abstraction degrees, for whichthe ratio is equal to or greater than the first threshold value and isequal to or less than the second threshold value, to the image.

It is preferable that the image processing device further comprises anabstraction degree setting unit that sets abstraction degrees of the taginformation items assigned to each image based on the tag informationitems assigned to the images included in the image set. It is preferablethat in a case where first tag information and second tag informationare tag information items included in the same category and the secondtag information is assigned to the image to which the first taginformation is assigned and the first tag information is not assigned tothe image to which the second tag information is assigned is equal to orgreater than a threshold value, the abstraction degree setting unit mayperform setting such that the abstraction degrees of the first taginformation and the second tag information are different and theabstraction degree of the second tag information is higher than theabstraction degree of the first tag information. It is preferable thatthe tag information setting unit sets the tag information items to beassigned to each image by using the tag information items which areassigned to the images included in the image set and for which theabstraction degrees are set, as the plurality of tag information itemshaving different abstraction degrees.

It is preferable that in a case where there are two or more taginformation items, among the tag information items to be assigned to theimage, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,the tag information assigning unit assigns, as the main tag information,the two or more tag information items for which the ratio is equal to orgreater than the first threshold value and is equal to or less than thesecond threshold value, to the image, for each image.

It is preferable that in a case where there are two or more taginformation items, among the tag information items to be assigned to theimage, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,the tag information assigning unit assigns, as the main tag information,only the tag information, of the two or more tag information items forwhich the ratio is equal to or greater than the first threshold valueand is equal to or less than the second threshold value, of which theabstraction degree is the maximum, to the image, for each image.

It is preferable that in a case where there are two or more taginformation items, among the tag information items to be assigned to theimage, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,the tag information assigning unit assigns, as the main tag information,only the tag information, of the two or more tag information items forwhich the ratio is equal to or greater than the first threshold valueand is equal to or less than the second threshold value, of which theabstraction degree is the minimum, to the image, for each image.

It is preferable that in a case where there is no tag information, amongthe tag information items to be assigned to the image, for which theratio is equal to or greater than the first threshold value and is equalto or less than the second threshold value, the tag informationassigning unit assigns, as the main tag information, only the taginformation, among the tag information items for which the ratio is lessthan the first threshold value, of which the ratio is the maximum, tothe image, for each image.

It is preferable that the image processing device further comprises animage narrowing-down unit that narrows down the images included in theimage set to an image group matching a narrowing-down conditionaccording to a command to set the narrowing-down condition of theimages. It is preferable that the tag information assigning unitassigns, as the main tag information, the tag information, among the taginformation items to be assigned to the image, for which a ratio of thenumber of times of appearances of the tag information to be assigned tothe image to the total number of times of appearances of all the taginformation items to be assigned to all the images included in the imagegroup is equal to or greater than a third threshold value and is equalto or less than a fourth threshold value, to the image, for each imageincluded in the image group.

It is preferable that the image receiving unit receives an input of anew image owned by the first user, and adds the new image to the imageset, the image analyzing unit analyzes the new image, the taginformation setting unit sets tag information items to be assigned tothe new image based on an analyzing result of the new image, and the taginformation assigning unit assigns, as the main tag information, the taginformation, among the tag information items to be assigned to theimage, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,to the image, for each image included in the image set including the newimage.

It is preferable that the image processing device further comprises animage sharing unit that shares an image between the first user and thesecond user. It is preferable that in a case where an image owned by thesecond user is shared with the first user, the image receiving unit addsthe shared image as the image set owned by the first user.

It is preferable that the image processing device further comprises animage display unit that displays at least a part of the images includedin the image set on a display. It is preferable that the image displayunit displays the main tag information assigned to the image displayedon the display in association with the image displayed on the display.

It is preferable that the tag information setting unit sets, as the taginformation items to be assigned to each image, tag informationindicating an emotion of a person appearing in each image.

It is preferable that the tag information setting unit sets, as the taginformation items to be assigned to each image, tag informationindicating an emotion of a person who captures each image according to acommand to set an emotion of a photographer.

It is preferable that the tag information assigning unit assigns taginformation, among the tag information items to be assigned to theimage, which is other than the tag information assigned as the main taginformation to the image, to the image, for each image.

The present invention provides an image processing method comprisingreceiving, by an image receiving unit, an input of an image set owned bya first user, analyzing, by an image analyzing unit, each image includedin the image set, setting, by a tag information setting unit, taginformation items to be assigned to each image based on an analyzingresult of each image, and assigning, by a tag information assigningunit, as main tag information, tag information, among the taginformation items to be assigned to the image, for which a ratio of thenumber of times of appearances of the tag information to be assigned tothe image to the total number of times of appearances of all the taginformation items to be assigned to all the images included in the imageset is equal to or greater than a first threshold value and is equal toor less than a second threshold value, to the image, for each image.

Here, it is preferable that two or more tag information items which havedifferent abstraction, among a plurality of tag information items whichis included in the same category and has different abstraction degreesare set as the tag information items to be assigned to the image foreach image, and the tag information, of the two or more tag informationitems which are to be assigned to the image and have differentabstraction degrees, for which the ratio is equal to or greater than thefirst threshold value and is equal to or less than the second thresholdvalue, is assigned as the main tag information to the image.

It is preferable that the image processing method further comprisesnarrowing down, by an image narrowing-down unit, the images included inthe image set to an image group matching a narrowing-down conditionaccording to a command to set the narrowing-down condition of theimages. It is preferable that the tag information, among the taginformation items to be assigned to the image, for which a ratio of thenumber of times of appearances of the tag information to be assigned tothe image to the total number of times of appearances of all the taginformation items to be assigned to all the images included in the imagegroup is equal to or greater than a third threshold value and is equalto or less than a fourth threshold value, is assigned as the main taginformation to the image, for each image included in the image group.

It is preferable that the image processing method further comprisesreceiving, by the image receiving unit, an input of a new image owned bythe first user, and adding the new image to the image set, analyzing, bythe image analyzing unit, the new image, and setting, by the taginformation setting unit, tag information items to be assigned to thenew image based on an analyzing result of the new image. It ispreferable that the tag information, among the tag information items tobe assigned to the image, for which the ratio is equal to or greaterthan the first threshold value and is equal to or less than the secondthreshold value, is assigned as the main tag information to the image,for each image included in the image set including the new image.

It is preferable that the image processing method further comprisesdisplaying, by an image display unit, at least a part of the imagesincluded in the image set on a display. It is preferable that the maintag information assigned to the image displayed on the display isdisplayed in association with the image displayed on the display.

The present invention provides a non-transitory computer recordingmedium storing a program causing a computer to execute the steps of anyof the image processing methods.

The present invention provides a computer-readable recording mediumhaving a program causing a computer to execute the steps of any of theimage processing methods recorded thereon.

The present invention provides an image processing device comprising animage receiving unit that receives an input of an image set owned by afirst user, an image analyzing unit that analyzes each image included inthe image set, a tag information setting unit that sets tag informationitems to be assigned to each image based on analyzing result of eachimage, and a tag information assigning unit that assigns, as main taginformation, the tag information, among the tag information items to beassigned to the image, for which a ratio of the number of times ofappearances of the tag information to be assigned to the image to thetotal number of times of appearances of all the tag information items tobe assigned to all the images included in the image set is equal to orgreater than a first threshold value and is equal to or less than asecond threshold value, to the image, for each image. The imagereceiving unit, the image analyzing unit, the tag information settingunit, and the tag information assigning unit are constituted bydedicated hardware or processors that execute program.

It is preferable that the image processing device further comprises anabstraction degree setting unit that sets abstraction degrees of the taginformation items assigned to each image based on the tag informationitems assigned to the images included in the image set. It is preferablethat in a case where first tag information and second tag informationare tag information items included in the same category and the secondtag information is assigned to the image to which the first taginformation is assigned and the first tag information is not assigned tothe image to which the second tag information is assigned is equal to orgreater than a threshold value, the abstraction degree setting unitperforms setting such that the abstraction degrees of the first taginformation and the second tag information are different and theabstraction degree of the second tag information is higher than theabstraction degree of the first tag information. It is preferable thatthe tag information setting unit sets the tag information items to beassigned to each image by using the tag information items which areassigned to the images included in the image set and for which theabstraction degrees are set, as the plurality of tag information itemshaving different abstraction degrees. It is preferable that theabstraction degree setting unit is constituted by dedicated hardware ora processor that executes a program.

It is preferable that the image processing device further comprises animage narrowing-down unit that narrows down the images included in theimage set to an image group matching a narrowing-down conditionaccording to a command to set the narrowing-down condition of theimages. It is preferable that the tag information assigning unitassigns, as the main tag information, the tag information, among the taginformation items to be assigned to the image, for which a ratio of thenumber of times of appearances of the tag information to be assigned tothe image to the total number of times of appearances of all the taginformation items to be assigned to all the images included in the imagegroup is equal to or greater than a third threshold value and is equalto or less than a fourth threshold value, to the image, for each imageincluded in the image group. It is preferable that the imagenarrowing-down unit is constituted by dedicated hardware or a processorthat executes a program.

It is preferable that the image processing device further comprises animage sharing unit that shares an image between the first user and thesecond user. It is preferable that in a case where an image owned by thesecond user is shared with the first user, the image receiving unit addsthe shared image as the image set owned by the first user. It ispreferable that the image sharing unit is constituted by dedicatedhardware or a processor that executes a program.

It is preferable that the image processing device further comprises animage display unit that displays at least a part of the images includedin the image set on a display. It is preferable that the image displayunit displays the main tag information assigned to the image displayedon the display in association with the image displayed on the display.It is preferable that the image display unit is constituted by dedicatedhardware or a processor that executes a program.

According to the present invention, it is possible to assign the maintag information having a high necessity for the first user to the imageby assigning, as the main tag information, the tag information, amongthe tag information items to be assigned to the image, for which theratio is equal to or greater than the first threshold value and is equalto or less than the second threshold value, to the image, for each imageincluded in the image set owned by the first user. It is possible toassign the main tag information having an appropriate abstraction degreeto each image included in the image set in the image set owned by thefirst user for each image by setting, as the tag information items to beassigned to the image, two or more tag information items which havedifferent abstraction degrees, among the plurality of tag informationitems which are included in the same category and have differentabstraction degrees.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an embodiment showing a configuration of animage processing system according to an embodiment of the presentinvention.

FIG. 2 is a block diagram of an embodiment showing a configuration of aclient.

FIG. 3 is a conceptual diagram of an embodiment showing tag informationincluded in a category of “vehicle”.

FIG. 4 is a flowchart of an embodiment showing an operation of the imageprocessing system in a case where main tag information is assigned to animage included in an image set.

FIG. 5 is a flowchart of an embodiment showing an operation of the imageprocessing system in a case where the image set is narrowed down to animage group.

FIG. 6 is a flowchart of an embodiment showing an operation of the imageprocessing system in a case where a new image is input.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an image processing device, an image processing method, aprogram, and a recording medium according to an embodiment of thepresent invention will be described in detail based on preferredembodiments shown in the accompanying drawings.

FIG. 1 is a block diagram of an embodiment showing a configuration of animage processing system according to the embodiment of the presentinvention. An image processing system 10 shown in FIG. 1 comprises aserver 12, and a plurality of clients 14 connected to the server 12 viaa network 16.

The server 12 has a function of obtaining image sets owned by aplurality of users who uses the image processing system 10 and settingeach image included in the image set owned by each of the plurality ofusers so as to publish each image to any user, so as to publish eachimage to only a sharing partner, or so as not to publish each image tothe public. The number of servers 12 is not limited to one, but aplurality of servers may be used. The server is a workstation includinga control device, a storage device, and a communication device.

The client 14 is an image processing device according to the presentembodiment, and has a function of assigning tag information to eachimage included in the image set owned by the user. The client 14 is adesktop personal computer (PC), a laptop PC, a tablet PC, or a portableterminal such as a portable phone and a smartphone that includes acontrol device, an input device, a storage device, a communicationdevice, and a display.

FIG. 2 is a block diagram of an embodiment showing a configuration ofthe client. The client 14 shown in FIG. 2 comprises a command obtainingunit 18, an image receiving unit 20, an image analyzing unit 22, a taginformation setting unit 24, a tag information assigning unit 26, anabstraction degree setting unit 28, an image narrowing-down unit 30, animage display unit 32, an image sharing unit 34, a communication unit36, and a display 38.

Hereinafter, the client 14 owned by a first user will be described. Thefirst user is any one user of all users who use the image processingsystem 10. The user indicates a person who uses one account in the imageprocessing system 10, and normally corresponds to one natural person.However, the user may be two or more natural persons at the time ofsharing one account in a family. In the present embodiment, although theclient 14 of the first user will be described, the client 14 owned byanother user other than the first user has the same configuration andfunction.

In the client 14 of the first user, the command obtaining unit 18initially obtains various commands input from the first user by using aninput device such as a keyboard, a mouse, or a touch panel.

The first user inputs a command by using the input device while viewinga screen displayed on the display 38. The command input by the firstuser is not particularly limited, but the command includes a command todesignate the image set, a command to select an emotion of aphotographer, a command to select a threshold value, a command to set anarrowing-down condition of images, a command to display an image, acommand to select an image from images included in the image set, acommand to set a sharing partner of images, and a command to set theimage so as to publish the image to the public, so as to share theimage, or so as not to publish the image to the public.

Subsequently, the image receiving unit 20 receives an input of the imageset owned by the first user according to the command to designate theimage set. In other words, the image receiving unit 20 obtains the imagegroup owned by the first user.

The image set owned by the first user is not particularly limited, butan image set stored in the storage device of the client 14 of the firstuser, an image set recorded in an external recording medium such as auniversal serial bus (USB) memory, a secure digital (SD) memory, acompact disc (CD), a digital versatile disc (DVD), and a hard disk (HD),an image set stored in an online storage, and an image set posted to anSNS are included in the image set. The image includes a still image anda motion picture.

Accessory information such as attribute information and tag informationcan be assigned to each image included in the image set.

The attribute information is information indicating an attribute of theimage, and can be, for example, an exchangeable image file format(Exif). The Exif information is information regarding an imagingcondition of the image, and includes, for example, information such asan imaging date and time of the image, an imaging location (positionalinformation), a focal length, or exposure. The Exif information isassigned as header information of the image (image data) captured by adigital camera.

The tag information is a name indicating the attribute information ofthe image, a subject (a person and an object other than the person)appearing in the image, and a scene, an event, and an emotion read fromthe image. For example, in a case where an imaging year of the image is2018, the tag information of “2018” is assigned to the image. In a casewhere the imaging location of the image is Tokyo, the tag information of“Tokyo” is assigned to the image. In a case where “meat” and a pluralityof “persons” appear as the subjects in the image, the tag informationitems of “meat”, “person”, “get-together”, and “girl-only gathering” areassigned to the image. In a case where “person” appearing in the imagehas a smile, the tag information items of “smile” and “enjoy” areassigned to the image. The emotion can be measured by measuring anddetecting a heart rate, a respiration rate, a blood pressure, anelectrocardiogram, a body temperature, and sweating by a wearableterminal, and by measuring and detecting a degree of opening of thepupil by a goggle type terminal. The tag information can beautomatically assigned based on an analyzing result of the image by thetag information assigning unit 26 to be described below or can bemanually assigned to the image by the first user.

Subsequently, the image analyzing unit 22 analyzes each image includedin the image set owned by the first user.

The image analysis is not particularly limited, but includes brightnessdetermination, color evaluation, blur evaluation, analysis of attributeinformation, subject recognition for recognizing the subject appearingin the image, scene recognition (night view, sea, sandy beach, sky, . .. ), and event detection (sports meeting, wedding ceremony, graduationceremony, . . . ). The subject recognition includes face detection,person recognition (face recognition), and object recognition (meat,fruit, flower, or cake).

Subsequently, the tag information setting unit 24 sets the taginformation items to be assigned to each image based on the analyzingresult of each image.

The tag information setting unit 24 can set, as the tag informationitems to be assigned to each image, the attribute information of eachimage, the subject appearing in each image, the scene or event of eachimage, and the tag information indicating the emotion of the personappearing in each image. The tag information setting unit 24 can set, asthe tag information items to be assigned to each image, the taginformation indicating the emotion of the person who captures each imageaccording to the command to set the emotion of the photographer.

Subsequently, the tag information assigning unit 26 assigns, as main taginformation, the tag information, among the tag information items to beassigned to the image set by the tag information setting unit 24, forwhich a ratio (an assignment rate of the tag information in the imageset) of the number of times of appearances of the tag information to beassigned to the image to the total number of times of appearances of allthe tag information items to be assigned to all the images included inthe image set is equal to or greater than a first threshold value and isequal to or less than a second threshold value, to the image, for eachimage included in the image set. The tag information assigning unit 26may assign, to the image, tag information, among the tag informationitems to be assigned to the image, which is other than the taginformation assigned as the main tag information to the image, for eachimage.

In a case where the image is displayed on the display 38, the main taginformation is tag information to be displayed in association with theimage. For example, all the tag information items which include the maintag information and are assigned to the image can be used as anarrowing-down condition at the time of narrowing down the image.

As stated above, a display region of a display of a smartphone isrelatively narrow, and thus, it may be difficult to display all the taginformation items assigned to the image in association with the image onthe display. In contrast, only the main tag information of the image inassociation with the image is displayed, and thus, it is possible todisplay the tag information items so as to be easier to see even in acase where the display region of the display is narrow compared to acase where all the tag information items are displayed.

For example, a category of the tag information indicates a range inwhich the tag information items having the same feature are included,such as “vehicle”, “food”, and “drink”. For example, tag informationitems such as “train”, “plane”, and “car” are included in the categoryof the tag information of “vehicle”, as shown in FIG. 3. For example,tag information items such as “4000 train”, “series 3000 series train”,and “2000 series train” are included in the category of the taginformation of “train”.

An abstraction degree of the tag information indicates a degree to whichthe tag information is expressed as a higher-level concept. For example,in a case where the category of the tag information is “vehicle”,abstraction degrees of the tag information items of “vehicle”, “train”,and “4000 series train” are respectively a large value, a medium value,and a small value. In other words, the tag information items of“vehicle”, “train”, and “4000 series train” are respectively taginformation items of a higher-level concept, a middle-level concept, anda lower-level concept, and have a hierarchical relation (inclusionrelation).

The first threshold value and the second threshold value arerespectively a lower limit value and an upper limit value of the ratio.The first threshold value and the second threshold value are values setin advance in the image processing system 10, and may be fixed valuessuch as 20% and 50%, or may be variable values according to the imageset. Alternatively, the first threshold value and the second thresholdvalue may be set according to the command to set the threshold value.For example, the first user can set the first threshold value and thesecond threshold value by using a slider bar. The same is true of athreshold value, a third threshold value, and a fourth threshold valueto be described below.

Subsequently, the abstraction degree setting unit 28 sets theabstraction degree of the tag information assigned to each image basedon the tag information assigned to the image included in the image set.

For example, in the image processing system 10, even though a pluralityof tag information items having different abstraction degrees is notprepared in advance for each category, in a case where the taginformation is already assigned to each image included in the image setreceived by the image receiving unit, the abstraction degree settingunit 28 can set the abstraction degree of the tag information assignedto each image, that is, the hierarchical relation of the tag informationbased on the tag information already assigned to each image included inthe image set.

For example, it is considered that the tag information of “vehicle” isassigned to the image to which the tag information of “train” isassigned with high frequency. Meanwhile, the tag information of “train”may not be necessarily assigned to the image to which the taginformation of “vehicle” is assigned, and the tag information items of“plane” and “car” may be assigned. In a case where there are many suchsituations, the abstraction degree of the tag information of “vehicle”may be higher than the abstraction degree of the tag information of“train”.

Accordingly, in a case where the first tag information and the secondtag information are the tag information items included in the samecategory and a case where the second tag information is assigned to theimage to which the first tag information is assigned and the first taginformation is not assigned to the image to which the second taginformation is assigned is equal to or greater than a threshold value,the abstraction degree setting unit 28 sets that the abstraction degreesof the first tag information and the second tag information aredifferent and the abstraction degree of the second tag information ishigher than the abstraction degree of the first tag information. In thiscase, the tag information setting unit 24 can set the tag informationitems to be assigned to each image by using, as a plurality of taginformation items, the tag information which is assigned to the imageincluded in the image set and to which the abstraction degree is set bythe abstraction degree setting unit 28.

Subsequently, the image narrowing-down unit 30 narrows down the imagesincluded in the image set to an image group matching to thenarrowing-down condition according to the command to set thenarrowing-down condition of the image. The narrowing-down conditionincludes any keyword, tag information, and Exif information. Only oneimage or two or more images may be included in the image group.

Subsequently, the image display unit 32 displays at least a part of theimages included in the image set on the display 38 according to thecommand to display the image. For example, the image display unit 32displays the image included in the image set or the image included inthe image group narrowed down by the image narrowing-down unit 30 on thedisplay 38. The image display unit 32 can display the main taginformation assigned to the image displayed on the display 38 inassociation with the image displayed on the display 38.

Subsequently, the image sharing unit 34 shares images between the firstuser and a second user.

The second user is any one user or a plurality of users other than thefirst user of all the users who use the image processing system 10. Theimage sharing unit 34 sets the image selected according to the commandto select the image so as to publish the image to any user or so as notto publish the image to the public according to the command to set theimage so as to publish the image or so as not to publish the image tothe public. The image sharing unit 34 shares the image selectedaccording to the command to select the image with only the sharingpartner set according to the command to set the sharing partneraccording to the command to share the image.

Subsequently, the communication unit 36 is the communication device ofthe client 14, and transmits and receives various data items to and fromthe communication device of the server 12.

For example, the communication unit 36 uploads the image (image data)stored in the storage device of the client 14 to the server 12 from theclient 14 via the network 16, and downloads the image stored in thestorage device of the server 12, which is configured such as HDD (Harddisc drive), to the client 14 from the server 12 via the network 16.

Next, an operation of the image processing system 10 in a case where themain tag information is assigned to the image included in the image setwill be described with reference to a flowchart shown in FIG. 4.

In the client 14 owned by the first user, the image receiving unit 20initially receives an input of the image set owned by the first useraccording to the command to designate the image set (S1). The image setis stored in the storage device of the client 14. The image set isuploaded to the server 12 from the client 14 via the network 16, and arestored in the storage device of the server 12.

Subsequently, each image included in the image set is analyzed by theimage analyzing unit 22 (S2).

Subsequently, the tag information setting unit 24 sets the taginformation items to be assigned to each image based on the analyzingresult of each image (S3).

Subsequently, the tag information assigning unit 26 assigns, as the maintag information, the tag information, among the tag information items tobe assigned to the image, for which the ratio of the number of times ofappearances to the total number of times of appearances is equal to orgreater than the first threshold value and is equal to or less than thesecond threshold value, to the image, for each image (S4).

Accordingly, it is possible to assign the main tag information havinghigh necessity for the first user to each image included in the imageset in the image set owned by the first user.

Next, the assigning method of the main tag information using the taginformation assigning unit 26 will be described.

In a case where there are two or more tag information items, among thetag information items to be assigned to the image, for which the ratiois equal to or greater than the first threshold value and is equal to orless than the second threshold value, the tag information assigning unit26 can assign, as the main tag information, these two or more taginformation items, to the image, for each image. For example, in a casewhere there are the tag information items of “train” and “4000 seriestrain” for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,both the tag information items of “train” and “4000 series train” areassigned as the main tag information to the image.

In a case where a priority is given to the tag information having ahigher abstraction degree, the tag information assigning unit 26 mayassign, as the main tag information, the tag information assigning unitassigns, as the main tag information, only the tag information, of thetwo or more tag information items for which the ratio is equal to orgreater than the first threshold value and is equal to or less than thesecond threshold value, of which the abstraction degree is the maximum,to the image. Similarly, in a case where there are the tag informationitems of “train” and “4000 series train” for which the ratio is equal toor greater than the first threshold value and is equal to or less thanthe second threshold value, the tag information assigning unit assigns,as the main tag information, only the tag information of “train” havinga higher abstraction degree to the image.

Alternatively, in a case where a priority is given to the taginformation having a lower abstraction degree, the tag informationassigning unit 26 may assign, as the main tag information, only the taginformation, of the two or more tag information items for which theratio is equal to or greater than the first threshold value and is equalto or less than the second threshold value, of which the abstractiondegree is the minimum, to the image. Similarly, in a case where thereare the tag information items of “train” and “4000 series train” forwhich the ratio is equal to or greater than the first threshold valueand is equal to or less than the second threshold value, the taginformation assigning unit assigns, as the main tag information, onlythe tag information of “4000 series train” having a lower abstractiondegree to the image.

Meanwhile, in a case where there is no tag information, among the taginformation items to be assigned to the image, for which the ratio isequal to or greater than the first threshold value and is equal to orless than the second threshold value, the tag information assigning unit26 may assign, as the main tag information, only the tag information,among the tag information items for which the ratio is less than thefirst threshold value, of which the ratio is the maximum, to the image,for each image. That is, among the tag information items for which thetag information is less than the first threshold value, the taginformation assigning unit assigns, as the main tag information, to theimage, the tag information having a higher ratio, that is, the taginformation having a higher degree of interest to the first user.

Immediately after the input of the image set is received by the imagereceiving unit 20, the analysis of the image, the setting of the taginformation, and the assignment of the tag information may be performed.Alternatively, after the input of the image set is received by the imagereceiving unit 20, the analysis of the image, the setting of the taginformation, and the assignment of the tag information may be performedfor a period for which any processing is not performed by the imageprocessing system 10, for example, in the nighttime. The input of notonly the image set owned by the first user but also the image sets ownedby two or more users who use the image processing system 10 may bereceived, and the analysis of the image, the setting of the taginformation, and the assignment of the main tag information may beperformed in the image sets owned by two or more users.

Here, it is considered that there are a smaller number of images towhich the tag information for which the ratio is less than the firstthreshold value is assigned and these images are the images having alower degree of interest to the first user among the image sets.Accordingly, there is a low necessity to assign, as the main taginformation, the tag information for which the ratio is less than thefirst threshold value to the image.

Meanwhile, it is considered that there are a large number of images towhich the tag information for which the ratio exceeds the secondthreshold value is assigned and these images are the images having ahigher degree of interest to the first user among the image set.However, since it is already considered to be known to the first userthat the tag information for which the ratio exceeds the secondthreshold value is assigned to a large number of images and there are alarge number of images to which the tag information for which the ratioexceeds the second threshold value is assigned, there is a low necessityto assign, as the main tag information, the tag information for whichthe ratio exceeds the second threshold value to the image.

For example, in a case where only an image of “vehicle” is included inthe image set owned by the first user, it is known to the first userthat each image is the image of “vehicle”, and there is a low necessityto assign, as the main tag information, the tag information of “vehicle”to the image. In other words, there is a high necessity to assign, asthe main tag information, the tag information of “train” or “4000 seriestrain” other than “vehicle” to the image for the first user.

It is considered that the image to which the tag information for whichthe ratio is equal to or greater than the first threshold value and isequal to or less than the second threshold value is assigned is theimage a higher degree of interest to the first user among the image set.Since it is considered not to be known to the first user that there isthe image to which the tag information for which the ratio is equal toor greater than the first threshold value and is equal to or less thanthe second threshold value is assigned, the tag information for whichthe ratio is equal to or greater than the first threshold value and isequal to or less than the second threshold value is set as the main taginformation, and thus, it is possible to assign the main tag informationhaving a high necessity for the first user to the image.

For example, in a case where only the image of “vehicle” is included inthe image set owned by the first user, it is assumed that the ratio ofthe tag information of “vehicle” exceeds the second threshold value, theratio of the tag information of “train” is equal to or greater than thefirst threshold value and is equal to or less than the second thresholdvalue, and the ratio of the tag information of “4000 series train” isless than the first threshold value. In this case, the tag informationof “train” is assigned as the main tag information to the image, andthus, it is possible to assign the main tag information having a highernecessity for the first user to the image.

In the present embodiment, the tag information setting unit 24 sets, asthe tag information items to be assigned to the image, two or more taginformation items which have different abstraction degrees, among theplurality of tag information items which is included in the samecategory and has different abstraction degrees, for each image. In thiscase, the tag information assigning unit 26 assigns, as the main taginformation, the tag information, of two or more tag information itemswhich are to be assigned to the image and have different abstractiondegrees, for which the ratio of the number of times of appearances tothe total number of times of appearances is equal to or greater than thefirst threshold value and is equal to or less than the second thresholdvalue, to the image. That is, the tag information setting unit 24 setsthe tag information items to be assigned to each image, among theplurality of tag information items which is prepared in advance in theimage processing system 10 and has different abstraction degrees foreach category.

Accordingly, it is possible to assign the main tag information having anappropriate abstraction degree to each image included in the image setin the image set owned by the first user.

Subsequently, the image display unit 32 displays the image included inthe image set on the display 38 according to the command to display theimage, and displays the main tag information assigned to the image inassociation with the image displayed on the display 38 (S5).

Accordingly, the first user can display the image included in the imageset on the display 38, and can confirm the image displayed on thedisplay 38 and the main tag information having an appropriateabstraction degree assigned to the image.

Next, an operation of the image processing system 10 in a case where theimages included in the image set are narrowed down to the image groupwill be described with reference to a flowchart shown in FIG. 5.

In this case, the image narrowing-down unit 30 narrows down the imagesincluded in the image set to the image group matching the narrowing-downcondition according to the command to set the narrowing-down conditionof the images (S6).

Subsequently, the tag information assigning unit 26 assigns, as the maintag information, the tag information, among the tag information items tobe assigned to the image, for which the ratio of the number of times ofappearances of the tag information to be assigned to the image to thetotal number of times of appearances of all the tag information items tobe assigned to all the images included in the image group is equal to orgreater than the third threshold value and is equal to or less than thefourth threshold value, to the image, for each image included in theimage group (S7).

Accordingly, it is possible to assign the main tag information having ahigh necessity for the first user to each image included in the imagegroup in the narrowed-down image group.

The third threshold value and the fourth threshold value used in theimage group correspond to the first threshold value and the secondthreshold value used in the image group. As stated above, in a casewhere a population of the images used at the time of assigning the maintag information is changed, the threshold value may be changed accordingto the population. In a case where the population of the images used atthe time of assigning the main tag information is changed, the taginformation assigned to each image is also changed as the main taginformation.

Subsequently, the image display unit 32 displays the image included inthe image group narrowed down by the image narrowing-down unit 30 on thedisplay 38, and displays the main tag information assigned to the imagein association with the image displayed on the display 38 (S8).

Accordingly, the first user can display the image included in the imagegroup narrowed down by the narrowing-down condition on the display 38,and can confirm the image displayed on the display 38 and the main taginformation having an appropriate abstraction degree assigned to theimage. Although not shown, in a case where the first user selects andviews one image displayed on the display 38, since it is considered thata display space relatively remains on the display 38, it is preferablethat other all tag information items are displayed in addition to themain tag information.

Next, an operation of the image processing system 10 in a case where anew image is input will be described with reference to a flowchart shownin FIG. 6.

In this case, the image receiving unit 20 receives an input of the newimage, and adds the new image to the image set owned by the first user(S9).

Subsequently, the image analyzing unit 22 analyzes the new image (S10).

Subsequently, the tag information setting unit 24 sets tag informationitems to be assigned to the new image based on an analyzing result ofthe new image (S11).

Subsequently, the tag information assigning unit 26 assigns, as the maintag information, the tag information, among the tag information items tobe assigned to the image, for which the ratio of the number of times ofappearances to the total number of times of appearances is equal to orgreater than the first threshold value and is equal to or less than thesecond threshold value, to the image, for each image included in theimage set which includes the new image and is owned by the first user(S12).

In a case where the new image is input, since the population of theimages used at the time of assigning the main tag information ischanged, the tag information to be assigned to each image is alsochanged as the main tag information. For example, in a case where ahobby of the first user is “train”, there are many images of “train”,but the hobby is changed to “plane”, a percentage of the images of“plane” occupying the image set is gradually increased. Accordingly, thetag information to be assigned as the main tag information to each imageis also changed.

Subsequently, the image display unit 32 displays the image included inthe image set which includes the new image and is owned by the firstuser on the display 38, and displays the main tag information assignedto the image in association with the image displayed on the display 38(S13).

Accordingly, the first user can display the image included in the imageset which includes the new image and is owned by the first user on thedisplay 38, and can confirm the image displayed on the display 38 andthe main tag information having an appropriate abstraction degreeassigned to the image.

In the image processing system 10, the first user and the second usersmay share the image by the image sharing unit 34.

For example, in a case where images owned by the second user are sharedwith the first user, the image receiving unit 20 can add the sharedimages as the image set owned by the first user. In this case, theshared images are also similarly used as the image set owned by thefirst user, and the processing such as the analysis of the image, thesetting of the tag information, the assignment of the main taginformation, and the display of the image and the main tag informationis performed.

Accordingly, the first user can display the image included in the imageset which includes the shared images and is owned by the first user onthe display 38, and can confirm the image displayed on the display 38and the main tag information having an appropriate abstraction degreeassigned to the image.

The image processing device according to the embodiment of the presentinvention may not be realized by only the client 14. The server 12 maycomprise one or more components of a plurality of components of theclient 14.

In the device according to the embodiment of the present invention, ahardware configuration of a processing unit that performs variousprocessing of the command obtaining unit 18, the image receiving unit20, the image analyzing unit 22, the tag information setting unit 24,the tag information assigning unit 26, the abstraction degree settingunit 28, the image narrowing-down unit 30, the image display unit 32,the image sharing unit 34, and the communication unit 36 may bededicated hardware, or may be various processors or computers thatperform programs, which is store in a recording medium.

Various processors include a central processing unit (CPU) which is ageneral purpose processor functioning as various processing units byexecuting software (program), a programmable logic device (PLD) which isa processor capable of changing a circuit configuration after a fieldprogrammable gate array (FPGA) is manufactured, and a dedicated electriccircuit which is a processor having a circuit configuration designed asa dedicated circuit in order to perform specific processing such asapplication specific integrated circuit (ASIC).

One processing unit may be constituted by one of the various processors.One processing unit may be obtained by combining the same kind ordifferent kinds of two or more processors, for example, by combining aplurality of FPGAs, or may be obtained by combining the FPGA and theCPU. The plurality of processing units may be constituted by one of thevarious processors or may be constituted by using one processor obtainedby combining two or more of the plurality of processing units.

For example, one processor is constituted by combining one or more CPUsand software so as to be represented by the computers such as the serverand the client, and this processor functions as the plurality ofprocessing units. A processor that realizes the functions of the entiresystem including the plurality of processing units by using oneintegrated circuit (IC) chip is used so as to be represented by a systemon chip (SoC).

More specifically, the hardware configuration of these variousprocessors is an electric circuitry obtained by combining circuitelements such as semiconductor elements.

For example, a method according to the embodiment of the presentinvention can be implemented by a program causing a computer to executesteps. It is possible to provide a computer-readable recording mediumhaving the program recorded thereon.

While the present invention has been described in detail, the presentinvention is not limited to the embodiment, and may be modified andchange in various manners without departing the gist of the presentinvention.

EXPLANATION OF REFERENCES

-   -   10: image processing system    -   12: server    -   14: client    -   16: network    -   18: command obtaining unit    -   20: image receiving unit    -   22: image analyzing unit    -   24: tag information setting unit    -   26: tag information assigning unit    -   28: abstraction degree setting unit    -   30: image narrowing-down unit    -   32: image display unit    -   34: image sharing unit    -   36: communication unit    -   38: display

What is claimed is:
 1. An image processing device comprising: an imagereceiving unit that receives an input of an image set owned by a firstuser; an image analyzing unit that analyzes each image included in theimage set; a tag information setting unit that sets tag informationitems to be assigned to each image based on analyzing result of eachimage; a tag information assigning unit that assigns, as main taginformation, the tag information, among the tag information items to beassigned to the image, for which a ratio of the number of times ofappearances of the tag information to be assigned to the image to thetotal number of times of appearances of all the tag information items tobe assigned to all the images included in the image set is equal to orgreater than a first threshold value and is equal to or less than asecond threshold value, to the image, for each image; and an abstractiondegree setting unit that sets abstraction degrees of the tag informationitems assigned to each image based on the tag information items assignedto the images included in the image set, wherein the tag informationsetting unit sets, as the tag information items to be assigned to theimage, two or more tag information items which have differentabstraction degrees, among a plurality of tag information items which isincluded in the same category and has different abstraction degrees, foreach image, the tag information assigning unit assigns, as the main taginformation, tag information, among the two or more tag informationitems which are to be assigned to the image and have differentabstraction degrees, for which the ratio is equal to or greater than thefirst threshold value and is equal to or less than the second thresholdvalue, to the image, in a case where first tag information and secondtag information are tag information items included in the same categoryand the second tag information is assigned to the image to which thefirst tag information is assigned and the first tag information is notassigned to the image to which the second tag information is assigned isequal to or greater than a threshold value, the abstraction degreesetting unit performs setting such that the abstraction degrees of thefirst tag information and the second tag information are different andthe abstraction degree of the second tag information is higher than theabstraction degree of the first tag information, and the tag informationsetting unit sets the tag information items to be assigned to each imageby using the tag information items which are assigned to the imagesincluded in the image set and for which the abstraction degrees are set,as the plurality of tag information items having different abstractiondegrees.
 2. The image processing device according to claim 1, wherein,in a case where there are two or more tag information items, among thetag information items to be assigned to the image, for which the ratiois equal to or greater than the first threshold value and is equal to orless than the second threshold value, the tag information assigning unitassigns, as the main tag information, the two or more tag informationitems for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,to the image, for each image.
 3. The image processing device accordingto claim 1, wherein, in a case where there are two or more taginformation items, among the tag information items to be assigned to theimage, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,the tag information assigning unit assigns, as the main tag information,only the tag information, of the two or more tag information items forwhich the ratio is equal to or greater than the first threshold valueand is equal to or less than the second threshold value, of which theabstraction degree is the maximum, to the image, for each image.
 4. Theimage processing device according to claim 1, wherein, in a case wherethere are two or more tag information items, among the tag informationitems to be assigned to the image, for which the ratio is equal to orgreater than the first threshold value and is equal to or less than thesecond threshold value, the tag information assigning unit assigns, asthe main tag information, only the tag information, of the two or moretag information items for which the ratio is equal to or greater thanthe first threshold value and is equal to or less than the secondthreshold value, of which the abstraction degree is the minimum, to theimage, for each image.
 5. The image processing device according to claim1, wherein, in a case where there is no tag information, among the taginformation items to be assigned to the image, for which the ratio isequal to or greater than the first threshold value and is equal to orless than the second threshold value, the tag information assigning unitassigns, as the main tag information, only the tag information, amongthe tag information items for which the ratio is less than the firstthreshold value, of which the ratio is the maximum, to the image, foreach image.
 6. The image processing device according to claim 1, furthercomprising: an image narrowing-down unit that narrows down the imagesincluded in the image set to an image group matching a narrowing-downcondition according to a command to set the narrowing-down condition ofthe images, wherein the tag information assigning unit assigns, as themain tag information, the tag information, among the tag informationitems to be assigned to the image, for which a ratio of the number oftimes of appearances of the tag information to be assigned to the imageto the total number of times of appearances of all the tag informationitems to be assigned to all the images included in the image group isequal to or greater than a third threshold value and is equal to or lessthan a fourth threshold value, to the image, for each image included inthe image group.
 7. The image processing device according to claim 1,wherein the image receiving unit receives an input of a new image ownedby the first user, and adds the new image to the image set, the imageanalyzing unit analyzes the new image, the tag information setting unitsets tag information items to be assigned to the new image based on ananalyzing result of the new image, and the tag information assigningunit assigns, as the main tag information, the tag information, amongthe tag information items to be assigned to the image, for which theratio is equal to or greater than the first threshold value and is equalto or less than the second threshold value, to the image, for each imageincluded in the image set including the new image.
 8. The imageprocessing device according to claim 1, further comprising: an imagesharing unit that shares an image between the first user and the seconduser, wherein, in a case where an image owned by the second user isshared with the first user, the image receiving unit adds the sharedimage as the image set owned by the first user.
 9. The image processingdevice according to claim 1, further comprising: an image display unitthat displays at least a part of the images included in the image set ona display, wherein the image display unit displays the main taginformation assigned to the image displayed on the display inassociation with the image displayed on the display.
 10. The imageprocessing device according to claim 1, wherein the tag informationsetting unit sets, as the tag information items to be assigned to eachimage, tag information indicating an emotion of a person appearing ineach image.
 11. The image processing device according to claim 1,wherein the tag information setting unit sets, as the tag informationitems to be assigned to each image, tag information indicating anemotion of a person who captures each image according to a command toset an emotion of a photographer.
 12. The image processing deviceaccording to claim 1, wherein the tag information assigning unit assignstag information, among the tag information items to be assigned to theimage, which is other than the tag information assigned as the main taginformation to the image, to the image, for each image.
 13. An imageprocessing method comprising: receiving, by an image receiving unit, aninput of an image set owned by a first user; analyzing, by an imageanalyzing unit, each image included in the image set; setting, by a taginformation setting unit, tag information items to be assigned to eachimage based on an analyzing result of each image; assigning, by a taginformation assigning unit, as main tag information, tag information,among the tag information items to be assigned to the image, for which aratio of the number of times of appearances of the tag information to beassigned to the image to the total number of times of appearances of allthe tag information items to be assigned to all the images included inthe image set is equal to or greater than a first threshold value and isequal to or less than a second threshold value, to the image, for eachimage; and setting, by an abstraction degree setting unit, abstractiondegrees of the tag information items assigned to each image based on thetag information items assigned to the images included in the image set,wherein two or more tag information items which have differentabstraction degrees, among a plurality of tag information items which isincluded in the same category and has different abstraction degrees areset as the tag information items to be assigned to the image for eachimage, the tag information, of the two or more tag information itemswhich are to be assigned to the image and have different abstractiondegrees, for which the ratio is equal to or greater than the firstthreshold value and is equal to or less than the second threshold value,is assigned as the main tag information to the image, in a case wherefirst tag information and second tag information are tag informationitems included in the same category and the second tag information isassigned to the image to which the first tag information is assigned andthe first tag information is not assigned to the image to which thesecond tag information is assigned is equal to or greater than athreshold value, the abstraction degrees are set such that theabstraction degrees of the first tag information and the second taginformation are different and the abstraction degree of the second taginformation is higher than the abstraction degree of the first taginformation, and the tag information items to be assigned to each imageare set as the plurality of tag information items having differentabstraction degrees by using the tag information items which areassigned to the images included in the image set and for which theabstraction degrees are set.
 14. The image processing method accordingto claim 13, further comprising: narrowing down, by an imagenarrowing-down unit, the images included in the image set to an imagegroup matching a narrowing-down condition according to a command to setthe narrowing-down condition of the images, wherein the tag information,among the tag information items to be assigned to the image, for which aratio of the number of times of appearances of the tag information to beassigned to the image to the total number of times of appearances of allthe tag information items to be assigned to all the images included inthe image group is equal to or greater than a third threshold value andis equal to or less than a fourth threshold value, is assigned as themain tag information to the image, for each image included in the imagegroup.
 15. The image processing method according to claim 13, furthercomprising: receiving, by the image receiving unit, an input of a newimage owned by the first user, and adding the new image to the imageset; analyzing, by the image analyzing unit, the new image; and setting,by the tag information setting unit, tag information items to beassigned to the new image based on an analyzing result of the new image,wherein the tag information, among the tag information items to beassigned to the image, for which the ratio is equal to or greater thanthe first threshold value and is equal to or less than the secondthreshold value, is assigned as the main tag information to the image,for each image included in the image set including the new image. 16.The image processing method according to claim 13, further comprising:displaying, by an image display unit, at least a part of the imagesincluded in the image set on a display, wherein the main tag informationassigned to the image displayed on the display is displayed inassociation with the image displayed on the display.
 17. Anon-transitory computer-readable recording medium having a programcausing a computer to execute the steps of the image processing methodaccording to claim 13 recorded thereon.
 18. An image processing devicecomprising: a processor configured to, receive an input of an image setowned by a first user; analyze each image included in the image set; settag information items to be assigned to each image based on analyzingresult of each image; assign, as main tag information, the taginformation, among the tag information items to be assigned to theimage, for which a ratio of the number of times of appearances of thetag information to be assigned to the image to the total number of timesof appearances of all the tag information items to be assigned to allthe images included in the image set is equal to or greater than a firstthreshold value and is equal to or less than a second threshold value,to the image, for each image; and set abstraction degrees of the taginformation items assigned to each image based on the tag informationitems assigned to the images included in the image set, wherein two ormore tag information items which have different abstraction degrees,among a plurality of tag information items which is included in the samecategory and has different abstraction degrees are set as the taginformation items to be assigned to the image for each image, the taginformation, of the two or more tag information items which are to beassigned to the image and have different abstraction degrees, for whichthe ratio is equal to or greater than the first threshold value and isequal to or less than the second threshold value, is assigned as themain tag information to the image, in a case where first tag informationand second tag information are tag information items included in thesame category and the second tag information is assigned to the image towhich the first tag information is assigned and the first taginformation is not assigned to the image to which the second taginformation is assigned is equal to or greater than a threshold value,the abstraction degrees are set such that the abstraction degrees of thefirst tag information and the second tag information are different andthe abstraction degree of the second tag information is higher than theabstraction degree of the first tag information, and the tag informationitems to be assigned to each image are set as the plurality of taginformation items having different abstraction degrees by using the taginformation items which are assigned to the images included in the imageset and for which the abstraction degrees are set.